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~ The MAILING DATE of this communication appears on the cover sheet with the correspondence address- 

All claims being allowable, PROSECUTION ON THE MERITS IS (OR REMAINS) CLOSED in this application. If not included 
herewith (or previously mailed), a Notice of Allowance (PTOL-85) or other appropriate communication will be mailed in due course. THIS 
NOTICE OF ALLOWABILITY IS NOT A GRANT OF PATENT RIGHTS. This application is subject to withdrawal from issue at the initiative 
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2. The allowed claim(s) is/are 1-3,5-9, 11-15,1 7-21,23 and 24 . 

3. ^ Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a) ^All b)DSome* c) □ None of the: 

1. ^ Certified copies of the priority documents have been received. 

2. □ Certified copies of the priority documents have been received in Application No. . 

3. □ Copies of the certified copies of the priority documents have been received in this national stage application from the 

International Bureau (PCT Rule 17.2(a)). 
* Certified copies not received: . 

Applicant has THREE MONTHS FROM THE "MAILING DATE" of this communication to file a reply complying with the requirements 
noted below. Failure to timely comply will result in ABANDONMENT of this application. 
THIS THREE-MONTH PERIOD IS NOT EXTENDABLE. 

4. □ A SUBSTITUTE OATH OR DECLARATION must be submitted. Note the attached EXAMINER'S AMENDMENT or NOTICE OF 

INFORMAL PATENT APPLICATION (PTO-152) which gives reason(s) why the oath or declaration is deficient. 

5. □ CORRECTED DRAWINGS ( as "replacement sheets") must be submitted. 

(a) □ including changes required by the Notice of Draftsperson's Patent Drawing Review ( PTO-948) attached 

1 ) □ hereto or 2) □ to Paper No./Mail Date . 

(b) □ including changes required by the attached Examiner's Amendment / Comment or in the Office action of 

Paper No./Mail Date . 

Identifying indicia such as the application number (see 37 CFR 1. 84(c)) should be written on the drawings in the front (not the back) of 
each sheet. Replacement sheet(s) should be labeled as such in the header according to 37 CFR 1.121(d). 

6. □ DEPOSIT OF and/or INFORMATION about the deposit of BIOLOGICAL MATERIAL must be submitted. Note the 

attached Examiner's comment regarding REQUIREMENT FOR THE DEPOSIT OF BIOLOGICAL MATERIAL. 



Attachment(s) 

1 . ^ Notice of References Cited (PTO-892) 

2. □ Notice of Draftperson's Patent Drawing Review (PTO-948) 

3. □ Information Disclosure Statements (PTO/SB/08), 

Paper No./Mail Date 

4. □ Examiner's Comment Regarding Requirement for Deposit 

of Biological Material 



5. □ Notice of Informal Patent Application 

6. □ Interview Summary (PTO-413), 

Paper No./Mail Date . 

7. ^ Examiner's Amendment/Comment 

8. ^ Examiner's Statement of Reasons for Allowance 

9. □ Other . 



/Brian P. Werner/ 

Supervisory Patent Examiner, Art Unit 2624 



U.S. Patent and Trademark Office 

PTOL-37 (Rev. 08-06) 



Notice of Allowability 



Part of Paper No./Mail Date 20090528 



Application/Control Number: 10/669,718 Page 2 

Art Unit: 2624 

EXAMINER'S AMENDMENT 

An examiner's amendment to the record appears below. Should the changes 
and/or additions be unacceptable to applicant, an amendment may be filed as 
provided by 37 CFR 1 .312. To ensure consideration of such an amendment, it 
MUST be submitted no later than the payment of the issue fee. 
Authorization for this examiner's amendment was given in a telephone interview 
with Mr. Jason Rhodes, on June 06 2009. 

The application has been amended as follows: 

1. (Currently Amended) A method of performing image processing on an image 
synthesized from a natural image and a computer graphic (CG) image that has a 
single color, said method comprising: 

utilizing [[at least one computer processor ]] a computer processor to 
perform the steps of: 

receiving via a computer input device designation of a region containing a 
natural-image region in the synthesized image; 

temporarily dividing the designated region into the natural-image region 
and a CG-image region by extracting from the designated region pixels that have 
the same color as the color contained in a region of the synthesized image other 
than the designated region, wherein 

when a first small region composed of pixels having the same color as the 
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color contained in the region other than the designated region is present in the 
designated region if the size of the first small region is greater than or equal to a 
threshold value, the first small region is extracted from the [[temporary ]] 
designated region to be classified as part of the CG-image region, and 

when a second small region composed of pixels having a color different 
from the color contained in the region other than the designated region is present 
in the designated region if the size of the second small region is greater than or 
equal to the threshold value, the second small region is not extracted from the 
designated region and is to be classified as part of the natural- image region; 

dividing said synthesized image into the natural-image region and the 
CG-image region; 

computing an image-processing parameter for said image processing, 
based on said natural-image region; 

acquiring an intermediate image by performing said image processing on 
said synthesized image, based on said image-processing parameter; and 

acquiring a processed image by synthesizing said natural-image region 
contained in said intermediate image and said CG-image region contained in 
said synthesized image. 

2. (Original) The method as set forth in claim 1, wherein a boundary portion 
between said natural-image region and CG-image region contained in said 
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synthesized image is blurred and then said CG-image region in said synthesized 
image and said natural-image region in said intermediate image are synthesized. 

3. (Original) The method as set forth in claim 1, wherein said synthesized image 
is obtained by reading out synthesized image data from a storage medium. 

4. (Canceled) 

5. (Previously Presented) The method as set forth in claim 1 , wherein said 
divided natural image and CG image are displayed. 

6. (Original) The method as set forth in claim 1 , wherein a maximum rectangular 
region that is inscribed in said natural-image region is set; and said image- 
processing parameter is computed based on an image within said maximum 
rectangular region. 

7. (Currently Amended) An apparatus for performing image processing on an 
image synthesized from a natural image and a computer graphic (CG) image that 
has a single color, said apparatus comprising: 

a computer processor which executes the following : 
a separation [[unit]] process for dividing said synthesized image into a 
natural-image region and a CG-image region, wherein 
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designation is received of a region containing the natural-image region in 
the synthesized image, 

the designated region is temporarily divided into the natural-image region 
and the CG-image region by extracting from the designated region pixels that 
have the same color as the color contained in a region of the synthesized image 
other than the designated region, wherein 

when a first small region composed of pixels having the same color as the 
color contained in the region other than the designated region is present in the 
designated region, if the size of the first small region is greater than or equal to a 
threshold value, the first small region is extracted from the designated region to 
be classified as part of the CG-image region, and 

when a second small region composed of pixels having a color different 
from the color contained in the region other than the designated region is present 
in the designated region, if the size of the second small region is greater than or 
equal to the threshold value, the second small region is not extracted from the 
designated region and is to be classified as part of the natural-image region, and 

said synthesized image is divided into the natural-image region and the 
CG-image region; 

a parameter computation [[unit]] process for computing an image- 
processing parameter for said image processing, based on said natural-image 
region; 
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a processing [[unit]] process for acquiring an intermediate image by 
performing said image processing on said synthesized image, based on said 
image-processing parameter; and 

a synthesis [[unit]] process for acquiring a processed image by 
synthesizing said natural-image region contained in said intermediate image and 
said CG-image region contained in said synthesized image. 

8. (Currently Amended) The [[image processor]] apparatus as set forth in claim 
7, wherein said synthesis [[unit]] process blurs a boundary portion between said 
natural-image region and CG-image region contained in said synthesized image 
and then synthesizes said CG-image region in said synthesized image and said 
natural-image region in said intermediate image. 

9. (Currently Amended) The [[image processor]] apparatus as set forth in claim 
7, further comprising a read-out [[unit]] process for obtaining said synthesized 
image by reading out synthesized image data from a storage medium. 

10. (Canceled). 

11. (Currently Amended) The [[image processor]] apparatus as set forth in claim 
7, further comprising a display screen for displaying said divided natural image 
and CG image. 
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12. (Currently Amended) The [[image processor]] apparatus as set forth in claim 
7, wherein said parameter computation [[unit]] process sets a maximum 
rectangular region that is inscribed in said natural-image region, and computes 
said image-processing parameter, based on an image within said maximum 
rectangular region. 

13. (Currently Amended) A system for performing image processing on an 
image synthesized from a natural image and a computer graphic (CG) image that 
has a single color, said system comprising: 

a computer input device configured to receive designation of a region 
containing a natural-image region in the synthesized image; and 
a computer processor programmed to : 

temporarily divide the designated region into the natural-image region and 
a CG-image region by extracting from the designated region pixels that have the 
same color as the color contained in a region of the synthesized image other 
than the designated region, wherein 

when a first small region composed of pixels having the same color as the 
color contained in the region other than the designated region is present in the 
designated region, if the size of the first small region is greater than or equal to a 
threshold value, the first small region is extracted from the designated region to 
be classified as part of the CG-image region, and 
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when a second small region composed of pixels having a color different 
from the color contained in the region other than the designated region is present 
in the designated region, if the size of the second small region is greater than or 
equal to the threshold value, the second small region is not extracted from the 
designated region and is to be classified as part of the natural-image region, 

divide said synthesized image into the natural-image region and a CG- 
image region, 

compute an image-processing parameter for said image processing, 
based on said natural-image region, 

acquire an intermediate image by performing said image processing on 
said synthesized image, based on said image-processing parameter, and 

acquire a processed image by synthesizing said natural-image region 
contained in said intermediate image and said CG-image region contained in 
said synthesized image. 

14. (Currently Amended) The system as set forth in claim 13, wherein [[said at 
least one computer processor]] said computer processor employs a procedure 
of blurring a boundary portion between said natural-image region and CG-image 
region contained in said synthesized image and then synthesizing said CG- 
image region in said synthesized image and said natural-image region in said 
intermediate image. 
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15. (Previously presented) The system as set forth in claim 13, further comprising 
a device configured to obtain said synthesized image by reading out synthesized 
image data from a storage medium. 

16. (Canceled) 

17. (Previously presented) The system as set forth in claim 13, further comprising 
a device configured to display said divided natural image and CG image. 

18. (Currently Amended) The system as set forth in claim 13, wherein said 
[[parameter computation procedure]] image- processing parameter computation 
is a procedure of setting a maximum rectangular region that is inscribed in said 
natural-image region, and computing said image-processing parameter, based 
on an image within said maximum rectangular region. 

19. (Currently Amended) A computer readable [[storage device]] recording 
medium having recorded therein a program for causing a computer to execute a 
method of performing image processing on an image synthesized from a natural 
image and a computer graphic (CG) image that has a single color, said program 
comprising: 

a procedure of receiving designation of a region containing a natural- 
image region in the synthesized image; 
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a procedure of temporarily dividing the designated region into the 
natural-image region and a CG-image region by extracting from the designated 
region pixels that have the same color as the color contained in a region of the 
synthesized image, other than the designated region; wherein 

when a first small region composed of pixels having the same color as 
the color contained in the region other than the designated region is present in 
the designated region if the size of the first small region is greater than or equal 
to a threshold value, the first small region is extracted from the designated region 
to be classified as part of the CG-image region, and 

when a second small region composed of pixels having a color different 
from the color contained in the region other than the designated region is present 
in the designated region if the size of the second small region is greater than or 
equal to the threshold value, the second small region is not extracted from the 
designated region and is to be classified as part of the natural-image region; 

a procedure of dividing said synthesized image into the natural-image 
region and the CG- image region; 

a procedure of computing an image-processing parameter for said image 
processing, based on said natural-image region; 

a procedure of acquiring an intermediate image by performing said 
image processing on said synthesized image, based on said image-processing 
parameter; and 

a procedure of acquiring a processed image by synthesizing said 
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natural-image region contained in said intermediate image and said CG-image 
region contained in said synthesized image. 

20. (Original)The computer readable recording medium as set forth in claim 19, 
wherein said synthesis procedure is a procedure of blurting a boundary portion 
between said natural- image region and CG-image region contained in said 
synthesized image and then synthesizing said CG-image region in said 
synthesized image and said natural-image region in said intermediate image. 

21. (Original) The computer readable recording medium as set forth in claim 19, 
wherein the program further comprises a procedure of obtaining said synthesized 
image by reading out synthesized image data from a storage medium. 

22. (Canceled) 

23. (Previously presented) The computer readable recording medium as set forth 
in claim 19, wherein the program further comprises a procedure of displaying 
said divided natural image and CG image. 

24. (Original) The computer readable recording medium as set forth in claim 19, 
wherein said parameter computation procedure is a procedure of setting a 
maximum rectangular region that is inscribed in said natural-image region, and 
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computing said image-processing parameter, based on an image within said 
maximum rectangular region. 

Reasons for allowance 

The following is an examiner's statement of reasons for allowance: None of the 
prior arts cited on PTO 892 disclose alone or in combination " temporarily 
dividing the designated region into the natural-image region and a CG-image 
region by extracting from the designated region pixels that have the same color 
as the color contained in a region of the synthesized image other than the 
designated region, wherein 

when a first small region composed of pixels having the same color as the 
color contained in the region other than the designated region is present in the 
designated region if the size of the first small region is greater than or equal to a 
threshold value, the first small region is extracted from the designated region to 
be classified as part of the CG-image region, and 

when a second small region composed of pixels having a color different 
from the color contained in the region other than the designated region is present 
in the designated region if the size of the second small region is greater than or 
equal to the threshold value, the second small region is not extracted from the 
designated region and is to be classified as part of the natural- image region; 

dividing said synthesized image into the natural-image region and the 
CG-image region; 
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computing an image-processing parameter for said image processing, 
based on said natural-image region; 

acquiring an intermediate image by performing said image processing on 
said synthesized image, based on said image-processing parameter" as recited 
in the independent claims 1,7,13 and 1 9, therefore Independent claims 1,7,13 
and 19 are allowed over the closest prior arts on record. Claims 2-3,5-9,1 1-15, 
17-21 and 23-24 depends directly or indirectly on the independent claims 1,7,13 
and 19 therefore they are allowed. 

Any comments considered necessary by applicant must be submitted no 
later than the payment of the issue fee and, to avoid processing delays, should 
preferably accompany the issue fee. Such submissions should be clearly labeled 
"Comments on Statement of Reasons for Allowance." 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 

examiner should be directed to JAYESH PATEL whose telephone number is 

(571)270-1227. The examiner can normally be reached on 5-4-9. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 

supervisor, Brian Werner can be reached on 571-272-7401 . The fax phone 

number for the organization where this application or proceeding is assigned is 

571-273-8300. 
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Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public 
PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll- 
free). If you would like assistance from a USPTO Customer Service 
Representative or access to the automated information system, call 800-786- 
91 99 (IN USA OR CANADA) or 571 -272-1 000. 

06/05/2009 
/JAYESH PATEL/ 
Examiner, Art Unit 2624 

/Brian P. Werner/ 

Supervisory Patent Examiner, Art Unit 2624 



